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BY TELEVISION# BECAUSE OF 

administering TEs»5 and .NVuNTORlES EY ItLtv t 

E-PENSE AND TECHNICAL DIFFICULTIES. DATA WERE COLLECTED IN A 
siHUCAtK SITUATION. THE SUOIECTS WERE 91 ELEVEN^-ORADE 
STUDENTS IN A FLORIDA HlOH SCHOOL. THE STUDENTS HERE 
administered the OCCUPATIONAL INTEREST INVENTORY <OII). 
ADVANCED FORM, UNDER NORMAL ADMINISTRATION CONDITIONS AND 
UNDER TWO EXPERIMENTAL CONDITIONS WHICH 

ADMINISTERIN6 THE Oil BY PROJECTINS “CH OF ITS ITMS ON A 3 
X 5- SCREEN ONE AT A TIME. THE ITEM EXI-OSURE TIME ON ONE 
EXPERIMENTAL ADMINISTRATION HAS E9UAL TO THE TIME *|;L*TE0 FOR 
THE on WHEN ADMINISTERED UNDER NORMAL CONDITIONS. CIVIDEO BY 

the total number of items, the item exposure time ^ the 

OTHER EXPERIMENTAL ADMINISTRATION MAS TWO- THIRDS THAT ^ THE 

former: and it was regarded as the 

ADDITION. SUBJECTS WERE ASKED TO A CRITERIA SCALE 

WHICH INVOLVED THEIR RANKING. ACCORDING TO PREFERENCE. THE 
OCCUPATIONS FOR WHICH SCORES ARE OBTAINED ON THE Oil. THE 
DATA REVEALED THAT THE CRITERION MEASURE WAS 

VALID. AN ANALYSIS OF THE BATA „ SIMILAR 

VOCATIONAL INTEREST INVENTORIES SUCH AS THE Oil YIELD SIMILAR 

SETS OF RANKINGS WHEN ITEMS ARE ADMINISTERED BY f««Jf«TING 
them OM a screen and when they are PRESENTED IN BOOKLET FORM. 
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A Conrpsrison of Scores Obtained 
Throu^ Normal and Visual Administrations of the 
Oc<nipational Interest Inventory^ 

H. W. Stoker, R. P. Kropp, and W. L. Bashaw 
i7irkvi<?n SH'.A^e Ifnlversity 




This 18 a report of one of a series of stndiSs vhich have heen designed In 
part to ascertain the feasihility of administering na television group tests 

and inventories of the type \Alch are commonly used In the public schools. 

« 

as the amount of instruction executed throujji the media of television 
is increasing it Is relevant to Inquire rtiether testing m^t a'so he done by 
television in such a way as to obtain the same, or perhaps more valid results 
than are afforded by the usual methods of administration. This report deals 
uith a description of an attempt to obtain, throu^i a simulated television 
adrinistratlon, results from an occupational interest inventory idilch are com- 
parable to those obtained under normal administrative conditions. 

Problem 

The problem dealt with here is a study of the relationships between cri- 
terion measures and sets of rankings of occupational groups for subjects (Ss) 
\iho were administered the s.'r'.e occupational interest inventory under three 
sets of conditions— one control end two e 3 Q?erimental. The control administra- 
tion (C) was conducted under those conditions recommended in the manual for the 



inventory; i.e., pencils, answer sheets, and booklets were distributed to 
subjects, the directions were read, etc. Both experimental administrations 

^ This study was supported by a grant from the 

Eaucation, National Defense Education Act of 1958, Title VII, Grant Runtoer 

7-08-075. 
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consisted of projecting the items of the inventory onto a screen one item at 
a time and the Ss recul and responded to the items as they appeared. The 
difference between the two es^erimentcJ. administrations was the intra-item 
exposure time. The intra-item exposure time for the experimental-normal (XN) 
administration was determined by alloting to each item twelve seconds ^ch 
is equivalent to dividing the time allowed for conqpleting the inventory under 
normal conditions by the number of items in the inventory. The intra-item 
esqposure time for the e 3 q>erimental -paced (XP) administration was determined by 
reducing to two-thirds the time alloted for each item in the (XN) administration. 

Although the major purpose of the study was to determine a feasible method 
of administering tests and inventories by television, due to the eaqpense and 
awkwardness of actually administering ej^erimentcCi tests by television, data 
were collected in a simulated situation. Items from the inventory were reproduced 
on slides and projected on a 3* x 5* screen. The slide format conformed to the 
format of a TV screen. The illumination level maintained in the testing room 
was comparable to that iseintalned for TV viewing. 

Criterion measures (CM) were collected from all They were presented 
with descriptions of Jobs in each of the six occupational categories included 
in the standardized inventory and were asked to indicate their preferences by 
ranking the six occupational groups. 

Procedure 

Subjects . The Ss consisted of all eleventh grade students in a \diite Florida 
high school. There were 91 Ss in all, two were discarded because they supplied 
inconq>lete data. 

Instrument. The Occupational Interest Inventory (Oil), Advanced Form, was used 
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in both normal and experimental administrations.^ 

^...„ental conditions , under the experimental condition the items from the 

on were presented one item per exposure. On the administration the item 

On the XP administration the item exposure 
exposure time was twelve seconds. On the «- earn. 

time was eight seconds. There was no inter-exposure interval. 

T^ntlns Schedule . The order in which tests were administrated was: CM, C, 

4. »nd two “word games" were used between administmUons 

XH, and XP. A rest period and two worn gamcD 

4 to reduce response set. The measures were 

of various conditions in an attenpt to reauce r pw 

obtained for all Ss during a four-hour period. 

fc, .11 8. »«. ««t«l "I" tb. » 

went the XP condition. 

« «» ““ “•’™“ “* 
"special" instructions about the mode of item presentation. Sa«g.le items were 
presented. Ss were not informed of the amount of time they would have for each 
item. They were informed that each item would be presented once and only once 
anring an administration. Ho questions were permitted when the test was in 

progress. 

No unusual circumstances arose during either the control or exper 
administrations . 

Results 

A coefficient of concordance (w) was computed on the ranks for each of the 
89 Ss Who completed the criterion scale and the C and XH administrations of the 
on. The distribution of these values appears in Table 1. Hote that Ss who 



1 Ihe Occupa^^ toterert^I^^to^^^ to^thfcSifS^Lt Bureau 

tSS; ^ci^s p;rmission to use the Oil for experimental purposes. 
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subsequently took the XP administration appear separately in the frequency 
column appeai*lng in the table* Each coefficient was tested for significance 
by the method outlined by Siegel.^ 

Insert Table 1 about here 

A coefficient of concordance was computed on the ranks for each of the 
forty-four Ss who completed, the CM and all three administrations of the Oil. 

The distribution of W's, each based on four sets of ranks, appears in Table 2* 

Insert Table 2 about here 

Rank order coefficients were computed between pairs of ranks on CM, C, 

XN and XP for each S. The distributions of these coefficients appear in Table 3» 

Insert Table 3 about here 

Sign tests were computed between rank correlations for various combina- 
tions of the four obtained measures. A summsiry of these comparisons appears 
in Table 4. 



Insert Table 4 about here 
Discussion 

The date in Table 1 reveal that 78 per cent of the V's computed on rank 
orders of occupations given by each subject on the CM, C, and XN administrations 
were equal to or greater than .80; and that 92 per cent of the W*s were 

^ Siegel, Sidney, Nonparametr i c Statistics for the Behavorial Sciences . New 
York: McGraw Hill, 1956. Chapter 9* 
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significant at least at the .05 level. The distribution of W's in Table 2, 
which were calculated on CM and the C, XN and XP administration of the Oil, 
reveal that 75 per cent of them were equal to or greater than .80; and that 
95 per cent were significant at least at the .05 level of confidence. Thus, 
one mig^t generalize that high relationships existed among the sets of occupa- 
tional ranks which ^ produced. 

The data in Table 3, which presents distributions of rho coefficients 
between padrs of rainks obtained under different administrative conditions, 
reveal that the following percentages of rho coefficients exceeded .80: between 

CM C, 5^ per cent; between CM and XN, 52 per cent; between CM and XP, 43 
per cent; between C and XN, 79 per cent; between C and XP, 79 per cent; and 
between XN auod XP, 77 per cent. Thus, it seems that CM rankings relate less 
strongly to the sets of rankings obtained by various administrations of the Oil 
than the latter sets relate to each other. Thus, one might conclude that rank- 
ings obtained by use of CM did not serve as good predictions of rankings obtained 
on the Oil which, due to its length and prior validation, was surely more relia- 
bel and valid. Consequently, one must asfume that the W’s which appear in 
Tables 1 and 2 underestimate the strength of relationship between various 
administrations of the Oil owing to the introduction of the somewhat irrelevant 
CM rcinkings. The percentsges reveal high relationship of rho’s that exceeded 
.80 suggest that the C, XN and XP administrations of the Oil are somewhat inter- 
changeable. 

The most pertinent sign test was the comparison of (C) (XN) with (c) (XP) 
which, although not significanl^ revealed that the coefficients between (C) (XP) 
were generally lower than those between (C) (XN). Inasmuch as the C administra- 
tion corresponded to the administration of the Oil under normal conditions, it 
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appears that of the two projected administrations of the Oil the XN administra- 
tion was superior. The item exposure interval on the XN administration was the 
same as that on the C administration if all Ss used all the time available for 
the C administration eind divided their time equally over all items. The XP 
administration, in which item exposure time was reduced 33 per cent from the XN 
administration, yielded rankings which generally related less strongly to C rank- 
ings for each S than did his XN rankings. 

The data and comments which appear above must be regarded as merely suggest- 
ive. On the one hand, no information was collected or available that dealt with 
the rho coefficients between rankings for normal administrations of the Oil; 
consequently, there is no backdrop against which to evaluate the obtained distri- 
butions of coefficients between C, X, and XN. On the other hand, all four sets 
of data were collected in a four -hour period. Data collected in such a short 
period mi^t be useful in determining the coefficient of equivalence for a test 
such as the Oil but such a coefficient is far less meaningful than a coefficient 
of stability because scores from such an instrument are commonly used in long- 
range vocational planning by the 

Summary 

The purpose of this study was to determine the relationships obtained by 
Ss who were administered the Occupational Interest Inventory under normal admin- 
istration conditions and two experimental conditions which involved administering 
the Oil by projecting one at a time each of its items on a 3* x 5* screen. The 

item ejqposure time on one eaqperiroental administration was equal to the time 

% 

alloted for the Oil when administered under normal conditions divided by the 
total number of items. The item exposure time on the other experimental admin- 
istration was two-thirds that of the former and it was regarded as the "paced" 
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administration. In addition Ss were asked to complete a criterion scale which 
involved their ranking, according to preference, the occupations for which scores 
are obtained on the Oil. The data revealed that the criterion measure was 
relatively invalid. 

Coefficients of concordance were computed on occupational rankings yielded 
on the criterion measure, and the control and normally-timed experimental admin- 
istrations; a separate set of coefficients was computed on the above sets of 
rankings and the rankings from the experimental -paced administration. The dis- 
tribution of W’s revealed that at least three-quarters of them were equal to or 
greater than .80, and approximately 90 cent of all W's were significant at 
least at the .05 level of confidence. The W's were regarded as underestimates 
of the relationships between Oil rankings obtained under various conditions 
because the rankings yielded by the criterion measure entered into the calcula- 
tion of W coefficients and it was subsequently learned that rankings on the 
criterion measure were not high3.y associated with Oil rankings. Rho coefficients 
between pairs of rankings revealed that 75 to 80 per cent of all coefficients 
confuted between pairs of control and the two e 3 q>erimental renkings were equal 
to or greater than .80. A sign test revealed that rho coefficients between the 
control and experimental normally-timed rankings were generally greater, but not 
significantly so, than the coefficients between the control and experimental paced 

rankings on the Oil. 

The analysis lends support to the belief tnat vocational interest inven- 
tories such as the Oil yield similar sets of rankings when the items are admin- 
istered by projecting them on a screen as when the items are presented in booklet 
form. No evidence was presented that dealt with the distribution of coefficients 
based on two normal administrations of the Oil; consequently the evaluation of 



Page 8 
Stoker 



obtained distributions of coefficients computed on ccntrol and experimental 
normally timed administrations, and control and experimental paced administra 
tions in terms of the distribution of coefficients between tv'o normal admin- 



istrations was not possible.] 
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Table 1. 

Distributions of Coefficients of Concordance for Ss 
on CM, and C and XN Administrations of Oil 










^ Ss who subsequently took XP administration 
^ Ss who did not take XP administration 



L 



ERIC 



W Interval®" 


Frequency^ 


Frequency*^ 


Total Frequency 


•95 - -99 


6 


10 


16 


.90 - . 9 ^ 


13 


7 


20 


.85 - .89 


12 


6 


18 


.80 - .Qk 


h 


11 


15 


•75 - -79 


1 


4 


5 


.70 - •7*» 


3 


4 


7 


.65 - •Sg 


1 


0 


1 


.60 - .6k 


1 


1 


2 


•55 - -59 


1 


0 


1 


.50 - -54 


1 


1 


2 


A5 - -49 


0 


0 


0 1 

1 


.40 - .44 


1 


0 


1 


•35 - -39 


0 


0 


0 


.30 - -34 


0 


1 


1 


^ W > .65 is significant at . 


,05 level 
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Table 2. 

Distributions of Coefficients of Concordance for Ss 
who Completed CM and C, XN, and XP Administrations of Oil 



W Interval®^ 


Frequency I 


.95 - -99 


4 1 


.90 - .9^ 


12 1 


.85 - .89 


10 1 


.80 - .84 


7 


.75 - .79 


5 1 


.70 - .74 


0 I 


.65 - .69 


1 1 


.60 - .64 


1 1 


.55 - .59 


2 1 


. .50 - .54 


1 1 


.45 - .49 


0 


.40 - .44 


1 1 


.35 - .39 




.30 - .34 





^ W > .55 significant at .05 level 
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Table 3 

Distributions of Rank Coefficients between 





Pairs of Ranks of Combinations of CM, C, XN, 


and XP 




— ■ ■■ — ■ ■ — ■ 

rho ■'•'tervad 






rho 


coefficients 






(CM)(C) 


(CM)(XN) 


(CM)(XP) 


(C)(XH) 


(C)(XP) 


(XH)(XP) 


1 -96 -1.00 


10 


6 


4 


40 


12 


24 


j .91 - .95 


11 


14 


5 


16 


11 


5 


! .86 - .90 


11 


13 


6 


11 


8 


2 


.81 - .85 


16 


13 


4 


3 


4 


3 


.76 ' .80 


4 


7 


6 


3 


4 


3 


.71 - .75 


9 


9 


2 


2 






.66 - .70 


3 


3 


1 


1 


1 


1 


.61 - .65 


5 


6 


4 


3 


2 


2 


.56 - .60 


7 


8 


4 


2 


1 




.51 - .55 


3 


1 


1 


2 


1 




.46 - .50 


5 


3 


2 




1 


1 


.41 - .45 


1 




1 


1 




2 


.36 - .40 


1 


2 


1 


1 






.31 - .35 






1 








.26 - .30 














.21 - .25 


1 


1 


1 


1 






.16 - .20 


1 


2 


1 








below .16 


1 


1 




3 


1 


2 



o 

ERIC 
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Table 4. 



Sign Tests for Selected Conparisons 



C. ■ M ■ - — — ‘ 

Comparison 




Number 


Differences 


i 

Probability 


CM vs. C, with CM vs. XN 


r >r 

(CM)(XN) (CM)(C) 


k2 




1 

1 


f 


r >r 

(CM)(C) (CM)(XN) 


38 


k 


.73 1 

I 


1 

! 

i 

1 

• 

1 


r =r 

(CM)(C) (CTI)(XH) 


9 




1 

! 

i 

» 

1 

i 

i 


CM vs. C, with CM vs. XP 


r >r 

(CM)(C) (CM)(XP) 


2k 




i 

j 

t 

i 

« 




r >r 

(CM)(XP) (CM)(C) 


17 


7 


i 

.35 




r ~T 

(CM)(XP) (CM)(C) 


3 




4 


C vs. XN, with C vs. XP 


(C)(XN) (C)(XP) 


22 








^(C)(XP) ^ (C)(XN) 


13 


9 


.18 




^O(XP) ^C)(XN) 


9 








